{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Forward matrix multiplication math for linear regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ p_{batch} = X_{batch} * W = \\begin{bmatrix}\n",
    "x_{11} & x_{12} & x_{13} & \\ldots & x_{1k} \\\\\n",
    "x_{21} & x_{22} & x_{23} & \\ldots & x_{2k} \\\\\n",
    "x_{31} & x_{32} & x_{33} & \\ldots & x_{3k}\n",
    "\\end{bmatrix} * \\begin{bmatrix} w_1 \\\\ w_2 \\\\ w_3 \\\\ \\vdots \\\\ w_k  \\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "x_{11} * w_1 + x_{12} * w_2 + x_{13} * w_3 + \\ldots + & x_{1k} * w_k \\\\\n",
    "x_{21} * w_1 + x_{22} * w_2 + x_{23} * w_3 + \\ldots + & x_{2k} * w_k \\\\\n",
    "x_{31} * w_1 + x_{32} * w_2 + x_{33} * w_3 + \\ldots + & x_{3k} * w_k\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "p_1 \\\\\n",
    "p_2 \\\\\n",
    "p_3\n",
    "\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ p_{batch\\_with\\_bias} = \n",
    "\\begin{bmatrix}\n",
    "x_{11} * w_1 + x_{12} * w_2 + x_{13} * w_3 + \\ldots + & x_{1k} * w_k + b \\\\\n",
    "x_{21} * w_1 + x_{22} * w_2 + x_{23} * w_3 + \\ldots + & x_{2k} * w_k + b\\\\\n",
    "x_{31} * w_1 + x_{32} * w_2 + x_{33} * w_3 + \\ldots + & x_{3k} * w_k + b\n",
    "\\end{bmatrix}\n",
    "=\n",
    "\\begin{bmatrix}\n",
    "p_1 \\\\\n",
    "p_2 \\\\\n",
    "p_3\n",
    "\\end{bmatrix} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ MSE(p_{batch}, y_{batch}) = MSE(\\begin{bmatrix}\n",
    "p_1 \\\\\n",
    "p_2 \\\\\n",
    "p_3\n",
    "\\end{bmatrix}, \\begin{bmatrix}\n",
    "y_1 \\\\\n",
    "y_2 \\\\\n",
    "y_3\n",
    "\\end{bmatrix}) = \\frac{(y_1 - p_1)^2 + (y_2 - p_2)^2 + (y_3 - p_3)^2}{3} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ p_i = x_i * W + b = w_1 * x_{i1} + w_2 * x_{i2} + \\ldots + w_k * x_{ik} + b $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sigmoid derivative"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ \\frac{\\partial \\sigma}{\\partial u}(x) = \\sigma(x) * (1 - \\sigma(x)) $$ "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
